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structure  for  evaluating  the  quality  of  combat  intelligence  data  was  under- 
taken. Fifty-six  officers  in  the  Intelligence  Officer's  Advanced  course 
were  split  into  two  groups;  one  group  rated  40  messages  from  the  Battle  of 
the  Bulge  scenario  used  in  the  original  research,  and  the  second  group 
rated  40  meesages  from  a scenario  based  on  a hypothetical  mid-intensity 
European  conflict.  Each  message  was  rated  on  50  rating  scales  previously  — 
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:ed,  as  well  as  on  four  new  scales  based  on  the  findings  of  the  previous 
research.  Factor  analyses  of  the  results  showed  that  the  structure  of 
ratings  within  both  scenarios  is  basically  the  same  as  that  derived  from 
the  enlisted  analysts.  Two  concepts,  TROTH  and  RELEVANCE/IMPORTANCE/THREAT, 
were  major  factors  in  the  evaluation  of  intelligence  information  in  all  cases . 
Two  of  the  new  scales,  "True"  and  "Predictable,"  reflect  the  TROTH  dimension, 
and  the  other  two  new  scales,  "Important"  and  "Relevant,"  represent  the 
RELEVANCE/IMPORTANCE/THREAT  dimension.  The  structure  of  intelligence 
information  quality  validated  in  the  present  study  is  not  well  represented 
by  the  current  ratings  of  Information  Accuracy  and  Reliability  of  Source. 

A new  Accuracy  scale  could  capture  a major  portion  of  the  variance  in 
quality  ratings,  and  the  other  new  scales  evaluated  here  provide  a basis 
for  further  development  of  supplementary  ratings. 
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ARI  Research  Reports  and  Technical  Papers  are  intended  for  sponsors  of 
R&D  tasks  and  other  research  and  military  agencies.  Any  findings  ready  for 
implementation  at  the  time  of  publication  are  presented  in  the  latter  part  of 
the  Brief.  Upon  completion  of  a major  phase  of  the  task,  formal  recommen- 
dations for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  is  concerned 
with  the  demands  of  the  future  battlefield  for  increased  man-machine 
complexity  to  acquire#  transmit#  process,  disseminate,  and  use  informa- 
tion. The  research  focuses  on  the  interface  problems  and  inter- 
actions within  command  and  control  centers  and  concerns  such  areas  as 
topographic  products  and  procedures,  tactical  symbology,  user-oriented 
systems,  information  management,  staff  operations  and  procedures,  and 
sensor  systems  integration  and  utilization. 

One  area  of  special  interest  is  the  evaluation  of  intelligence 
data  within  tactical  intelligence  systems.  Analysts  involved  in  the 
integration  of  data  from  many  sources,  as  well  as  decisionmakers,  can 
benefit  from  ratings  of  information  quality.  Current  doctrine  calls 
for  explicit  judgments  of  information  accuracy  and  source  reliability, 
but  there  is  strong  evidence  that  these  ratings  are  poorly  understood 
and  seldom  used  effectively. 

The  present  publication  is  a continuation  of  previous  research 
on  the  judgment  and  use  of  ratings  of  intelligence  information.  Tech- 
nical Paper  286,  which  suggested  ratings  on  two  or  three  new  dimensions, 
would  capture  most  of  the  important  aspects  of  information  quality. 

The  present  study  added  a second  scenario,  used  a different  population 
of  raters,  and  confirmed  the  earlier  findings.  Numerical  ratings  of 
Truth  or  Accuracy  and  of  Importance  may  provide  a good  alternative  to 
the  current  rating  scheme. 

Research  in  the  area  of  data  characteristics  and  information 
utilization  is  conducted  as  an  in-house  effort  augmented  by  contracts 
with  organizations  that  have  unique  capabilities  and  facilities  for 
research  in  this  area.  The  present  study  was  conducted  as  an  in-house 
effort  under  the  program  direction  of  Mr.  Robert  s.  Andrews.  The 
effort  was  responsive  to  requirements  of  Army  Project  2Q762722A765  and 
of  the  U.S.  Army  Intelligence  Center  and  School,  Fort  Huachuca,  Ariz. 
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A VALIDATION  OF  THE  STRUCTURE  OF  COMBAT  INTELLIGENCE  RATINGS 


BRIEF 


Requirement : 

To  validate  a multidimensional  descriptive  structure  of  the  qual- 
ity of  intelligence  data  and  to  evaluate  four  new  scales  for  combat 
intelligence — True,  Predictable,  Important,  and  Relevant — based  on 
the  descriptive  structure. 


Procedure: 

Fifty-six  captains  enrolled  in  the  Intelligence  Officers 1 Advanced 
Course  were  divided  into  two  groups  of  28  each.  Officers  in  one  group 
evaluated  a series  of  40  messages  based  on  the  Battle  of  the  Bulge,  and 
those  in  the  second  group  evaluated  a series  of  40  messages  based  on  a 
hypothetical  mid-intensity  conflict  in  west-central  Europe.  The  eval- 
uations were  made  on  50  descriptive  scales  that  had  been  used  in  an 
earlier  study  and  that  were  the  basis  of  the  descriptive  structure  being 
validated.  Subsets  of  20  messages  from  each  of  the  two  message  sets  also 
were  rated  on  four  scales  designed  to  represent  the  previously  identified 
descriptive  structure.  The  evaluations  were  analyzed  to  determine  the 
implicit  judgment  structure  underlying  the  officers'  ratings  and  to  eval- 
uate the  correspondence  of  the  four  new  rating  scales  with  this  judgment 
structure . 


Findings : 

There  were  no  significant  differences  between  judgments  in  the  two 
scenarios. 

Officers'  judgments  of  the  quality  of  intelligence  data  can  be 
represented  by  a multidimensional  structure  with  three  or  four 
dimensions. 

The  major  dimensions  of  intelligence  data  quality  were  not  signifi- 
cantly different  from  those  found  in  a previous  experiment  using  enlisted 
intelligence  analysts. 

Two  concepts,  TRUTH  and  RELEVANCE/ IMPORTANCE/THREAT,  were  major 
factors  for  evaluation  of  combat  intelligence  by  all  groups  of 
analysts. 


■ 


Two  of  the  new  scales  based  on  the  earlier  study.  True  and  Predict- 
able, reflect  the  TRUTH  dimension.  The  other  two  new  scales.  Important 
and  Relevant,  represent  the  RELEVANCE/ IMPORTANCE/THREAT  dimension. 


Utilization  of  Findings: 

The  implicit  structure  of  combat  intelligence  ratings,  which  was 
validated  in  this  research,  can  be  used  to  describe  the  evaluations  made 
by  a variety  of  intelligence  personnel  of  a variety  of  combat  intelli- 
gence data.  Thus,  this  structure  provides  a basis  for  improving  proce- 
dures for  evaluating  combat  intelligence  data. 

This  empirical  structure  is  not  well-defined  by  the  current 
Accuracy/Reliability  ratings.  The  four  new  scales,  especially  the 
True  scale,  provide  a basis  for  the  empirical  development  of  rating 
scales  that  are  more  natural  and  easier  to  use  than  existing  rating 
scales.  These  results  serve  to  focus  future  efforts  to  develop  rating 
scales  for  combat  intelligence  data  which  will  facilitate  the  effective 
utilization  of  the  data. 
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A VALIDATION  OF  THE  STRUCTURE  OF  COMBAT  INTELLIGENCE  RATINGS 


INTRODUCTION 

Intelligence  on  which  an  Army  commander  bases  decisions  typically 
passes  through  several  stages  of  analysis  and  integration  before 
arriving  in  the  form  of  conclusions  or  estimates.  The  initial  raw  data 
(for  example,  a reported  sighting  of  an  enemy  vehicle  moving  in  a given 
direction)  is  combined  with  other  related  data  to  give  a coherent  picture 
of  current  enemy  activity.  A critical  step  in  effective  data  utilization 
at  each  stage  of  the  intelligence  process  is  a determination  of  the 
"quality"  of  each  item  of  available  data. 

Current  Army  doctrine  (FM  30-5,  1973)  requires  the  explicit  rating 
of  intelligence  data  on  two  six-point  rating  scales:  Accuracy  of  Infor- 
mation and  Reliability  of  Source.  In  addition,  an  implicit  evaluation 
of  pertinence  and  timeliness  is  reflected  in  an  analyst's  decision  to 
examine  data.  A variety  of  field  and  laboratory  experiments  as  well  a: 
anecdotal  evidence  have  shown  that  the  current  Accuracy  and  Reliability 
scales  frequently  are  not  used  and  are  often  misapplied  or  misinterpreted 
when  used. 

An  explanation  for  deficiencies  in  the  use  of  the  two  current 
rating  scales  was  suggested  by  recent  ARI  research  (Miron,  Patten,  and 
Halpin,  1978) . It  appears  that  intelligence  analysts  do  not  evaluate/ 
perceive  the  "quality"  of  intelligence  data  in  terms  of  the  four  dimen- 
sions prescribed. by  current  Army  doctrine.  A somewhat  different  concep- 
tual structure  for  the  evaluation  of  intelligence  data  was  determined 
empirically  by  Miron  et  al.  (1978)  which,  if  valid,  could  provide  a 
basis  for  the  development  of  more  effective  procedures  for  rating  the 
quality  of  intelligence  data.  The  present  experiment  is  an  attempt  to 
validate  the  conceptual  structure  of  combat  intelligence  ratings  suggest- 
ed by  Miron  et  al.  (1978)  and  to  evaluate  several  candidate  rating  scales 
derived  from  this  conceptual  structure . 


Current  Quality  Ratings 

The  present  system  for  evaluating  the  quality  of  intelligence  data 
has  changed  little  since  World  War  II  (FM  30-5) . Upon  receipt  of  an 
intelligence  report,  the  analyst  must  first  determine  if  the  information 
is  pertinent  and  timely.  The  analyst  also  must  evaluate  the  reputation 
of  the  source  and/or  reporting  agency  for  submitting  factual  reports 
(Reliability)  as  well  as  the  factual  nature  (Accuracy)  of  the  data  being 
processed.  The  evaluations  of  pertinence  and  timeliness  presumably  are 
reflected  in  the  analyst's  decision  to  continue  an  examination  of  the 
data.  The  Accuracy  and  Reliability  evaluations  are  reflected  in  explicit 
judgments  on  two  standard  rating  scales.  These  ratings  then  become 
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attached  to  the  data,  and  other  users  of  the  data  determine  at  least 
in  part,  the  quality  and  usefulness  of  the  data  on  the  basis  of  these 
ratings . 

Reliability  is  judged  on  the  basis  of  a six-letter  system,  and 
Accuracy  is  judged  on  the  basis  of  a six-number  system.  Letters  and 
numbers  are  defined  by  an  adjective  or  adjectival  term  as  follows: 


Reliability  of  Source  and/or  Agency 
A - Completely  Reliable 
B - Usually  Reliable 
C - Fairly  Reliable 
D - Not  Usually  Reliable 
E - Unreliable 


Accuracy  of  Information 

1 - Confirmed  by  Other  Sources 

2 - Probably  True 

3 - Possibly  True 

4 - Doubtfully  True 

5 - Improbable 


F - Reliability  Cannot  Be  Judged  6 - Truth  Cannot  Be  Judged 

There  are  several  problems  with  the  present  system  of  evaluation. 
Analysts  seldom  agree  on  the  precise  meaning  of  the  descriptive  words 
and  terms.  Possibly  because  of  this  confusion,  the  ratings  are  fre- 
quently omitted  (Baker,  Mace,  and  McKendry,  1968).  In  addition,  although 
analysts  are  specifically  instructed  to  rate  information  Reliability  and 
Accuracy  as  separate  items , prior  research  has  revealed  that  individual 
analysts  consistently  use  either  Reliability  or  Accuracy  as  the  basic 
criterion  of  message  quality  and  correlate  the  other  rating  with  it 
(Samet,  1975;  Meehland  and  Rine,  1967) . As  a result,  ratings  tend  to 
run  A-l , B-2,  C-3...  Studies  also  have  indicated  that  most  analysts  use 
Accuracy  as  their  basic  criterion,  and  that  they  may  tend  to  inflate 
their  ratings;  nearly  three-quarters  of  all  reports  are  rated  B-2  (Samet, 
1975;  Baker  et  al.,  1968). 


The  laboratory  and  field  results  summarized  above  indicate  that 
the  current  system  for  evaluating  intelligence  data  does  not  provide 
an  effective,  realistic  evaluation  of  data  quality.  One  possible 
explanation  of  these  results  is  that  intelligence  analysts  neither 
perceive  nor  evaluate  information  quality  in  terms  of  Reliability  and 
Accuracy  as  defined  by  current  doctrine. 


Structure  of  Combat  Intelligence  Ratings 

The  approach  used  by  Miron  et  al.  (1978)  to  uncover  the  psycholog- 
ical bases  for  judgments  of  information  quality  was  to  have  a number  of 
trained  and  untrained  intelligence  analysts  make  many  distinct  ratings 
of  "quality"  for  each  of  40  intelligence  messages.  These  ratings  were 
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examined  for  patterns  that  indicate  the  underlying  criteria  being 
used  by  the  analysts.  An  initial  conceptual  analysis  of  the  intelli- 
gence data  evaluation  task  identified  the  following  six  types  of  evalu- 
ations that  might  be  made:  (1)  judgments  concerning  the  data  source; 

(2)  judgments  concerning  the  message  content;  (3)  judgments  concerning 
the  situation;  (4)  judgments  concerning  the  action  reported  considering 
the  overall  tactical  situation;  (5)  judgments  concerning  the  inferences 
that  can  be  drawn  from  the  data;  and  (6)  judgments  of  characteristics 
of  the  report  itself.  Forty-six  bipolar  scales  (e.g.,  Direct/Indirect; 

Widespread/Local;  Garbled/Clear)  were  generated  to  represent  these 
general  concepts;  these  scales,  the  standard  Reliability  and  Accuracy 
scales,  and  two  "global  validity"  scales  were  used  by  the  analysts  to 
evaluate  the  40  messages  (Figure  1) . The  "Global  Validity"  scales, 
suggested  by  Samet  (1975) , require  a rating  on  a 0-100  scale  of  the 
general  validity  or  accuracy  of  the  data  in  the  message. 

The  message  set  used  consisted  of  33  messages  from  the  files  of 
the  28th  Infantry  Division  for  the  period  10  December  through  15 
December  1944  (just  before  the  German  Ardennes  Counteroffensive  of 
16  December  1944) , and  an  additional  seven  fictitious  and  misleading 
messages  generated  for  the  research.  The  raters  were  21  "trained" 
enlisted  personnel  who  had  just  completed  the  course  for  intelligence 
analysts  (96B)  at  the  U.S.  Army  Intelligence  Center  and  School  (USAICS) 
and  34  "untrained"  enlisted  personnel  who  were  just  entering  .the  same 
course.  Most  of  the  participating  personnel  had  little  Army  experience 
beyond  Basic  Training  and,  in  the  case  of  the  "trained"  group,  the 
USAICS  course. 

The  ratings  of  the  messages  on  the  50  scales  were  analyzed  to 
determine  the  underlying  cognitive  structure  that  would  explain  the 
pattern  of  relationships  among  the  various  scales.  Scales  that  were 
used  in  a similar  fashion  across  messages  were  assumed  to  have  a 
common  basis;  thus,  for  example,  if  Messages  1,  3,  9,  17,  and  25  all 
were  rated  "high"  on  scales  X,  Y,  and  Z,  and  if  all  other  messages 
were  rated  "low"  on  these  scales,  there  is  a strong  indication  that 
scales  X,  Y,  and  Z overlap  in  their  meaning  and  all  represent  one 
under lying  concept. 

The  data  analyses  were  designed  to  identify  the  most  parsimonious 
set  of  concepts  or  dimensions  that  would  describe  the  obtained  pattern 
of  scale  interrelationships.  The  analysis  of  the  participants'  ratings 
indicated  that  there  were  no  basic  differences  between  the  two  groups, 
and  that  there  were  three  basic  "factors"  or  "dimensions"  in  the  psycho- 
logical space  that  determined  the  way  the  selected  intelligence  messages 
were  evaluated  (Table  1) . In  other  words,  the  analysts  (both  trained 
and  untrained)  essentially  made  only  three  independent  evaluations  of 
each  message  in  using  the  50  rating  scales . 
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Figure  1.  Rating  scales  used  to  evaluate  data  elements 


The  primary  factor  concerned  the  TRUTH,  or  accuracy,  of  the 
message.  Evaluations  of  this  aspect  of  message  quality  were  related 
to  the  current  Accuracy  and  Reliability  scalef,  the  Global  Validity 
scales,  and  bipolar  scales  such  as  True/False,  Probable/Improbable, 
and  Acceptable/Unacceptable. 

The  ratings  that  defined  the  second  factor  suggested  that  the 
analysts  were  basing  many  of  their  ratings  on  some  concept  of  informa- 
tion RELEVANCE,  or  importance;  evaluations  of  this  aspect  of  message 
quality  were  related  to  bipolar  scales  such  as  Heavy/Light,  Many/Few, 
Large-Scale/Small-Scale,  and  Risky/Routine. 

The  third  aspect  of  message  quality  was  identified  tentatively 
as  DIRECTNESS;  here  the  evaluations  were  related  to  bipolar  scales 
such  as  Interpreted/Uninterpreted,  Implied/Unimplied,  and  Under- 
standable/Confusing . 

Given  the  limited  experience  of  the  analysts  in  the  Miron  et  al. 
(1978)  study  and  the  possible  limitations  imposed  by  the  use  of  a 
scenario  reflecting  warfare  conditions  30  years  ago  (e.g.,  reports  of 
sightings  of  horsedrawn  carts) , these  results  nevertheless  suggest  that 
several  modifications  in  the  procedures  used  to  evaluate  intelligence 
data  may  be  desirable.  As  in  earlier  research,  analysts  did  not  dis- 
criminate between  information  Accuracy  and  information  Reliability;  both 
of  these  ratings  appear  to  be  used  to  represent  some  notion  of  Accuracy. 
Furthermore,  this  underlying  concept  of  information  Accuracy  may  be 
better  represented  by  the  0-100  scale  of  Global  validity  than  by  either 
of  the  two  standard  scales.  Validation  of  these  results  with  a sample 
of  more  experienced  intelligence  personnel  and  with  a different  scenario 
would  indicate  that  the  two  current  six-point  scales  for  the  evaluation 
of  information  quality  should  be  replaced.  Validation  would  also  provide 
a basis  for  developing  rating  scales  to  measure  aspects  of  information 
quality  that  are  more  consistent  with  the  analyst's  subjective  appraisal 
and  the  needs  of  the  intelligence  system. 


The  conceptual  structure  of  combat  intelligence  ratings  developed 
by  Miron  et  al.  (1978)  can  serve  as  the  basis  for  improved  rating  scales 
to  evaluate  combat  intelligence  data.  Determining  the  types  of  rating 
scales  that  should  be  used  requires  an  iterative  procedure  of  developing, 
evaluating,  and  refining  new  rating  scales. 

Four  potential  scales  for  evaluating  combat  intelligence  data 
(Figure  2)  were  designed  to  provide  ratings  which  correspond  to  the 
three  factors  found  by  Miron  et  al.  (1978);  Scale  1 focuses  on  TRUTH, 
Scales  2 and  4 focus  on  RELEVANCE/ IMPORTANCE/THREAT,  and  Scale  3 
focuses  on  DIRECTNESS . Numerical  scales  were  used  because  any  single 
key  word  or  word  pair  is  unlikely  to  unambiguously  define  the  underlying 
concept  represented  by  any  of  the  judgment  dimensions.  Furthermore, 
numerical  scales  have  several  distinct  advantages:  Values  on  a scale 
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Figure  2.  Four  potential  scales  for  evaluating  combat 

intelligence  data. 


can  be  expressed  clearly  and  unambiguously,  they  are  easily  scorable, 
and  they  can  be  used  in  a variety  of  presentation  forms  (e.g.,  charts, 
graphs,  tables) . Although  simple,  these  four  scales  are  a starting 


point  for  the  empirical  development  of  improved  evaluation  procedures 
for  tactical  intelligence. 


Objectives 

1.  To  validate  the  structure  of  combat  intelligence  ratings  devel- 
oped empirically  by  Miron  et  al.  (1978)  by  replicating  the  experiment 
using  experienced  intelligence  personnel  and  a different  scenario. 

2.  To  evaluate  the  effectiveness  of  four  new  rating  scales  designed 
to  measure  the  primary  factors  of  the  Miron  et  al.  structure  of  combat 
intelligence  ratings. 


METHOD 

Participants 

Fifty-six  officers  enrolled  in  the  Intelligence  Officers  Advanced 
Course  at  the  U.S.  Army  Intelligence  Center  and  School,  Fort  Huachuca, 
Ariz.,  participated  in  this  experiment.  They  were  divided  into  two 
equal  groups,  each  of  which  received  materials  from  a different  scenario. 
The  groups  were  equivalent  in  rank  and  intelligence  experience.  One 
group  had  27  0-3' s and  1 0-2,  and  the  other  had  28  0-3' s.  Members 
of  one  group  had  from  1 to  11  years  of  intelligence  experience,  with 
a mean  of  3.8  years,  and  the  other  group’s  members  had  from  1 to  10 
years  of  intelligence  experience  with  a mean  of  3.9  years. 


Scenarios 

Two  scenarios  were  used.  The  first  was  based  on  the  Battle  of  the 
Bulge  and  was  identical  to  that  used  in  the  previous  research  by  Miron 
et  al.  (1978). 

Thirty-three  intelligence  messages  concerning  enemy  activity  during 
the  period  10  December  through  15  December  1944  along  the  Ardennes  line 
in  Europe  were  drawn  from  the  files  of  the  28th  Infantry  Division.  Seven 
fictitious  and  misleading  reports  were  added  to  these  messages  to  deter- 
mine if  analysts  would  detect  them.  These  messages  (Miron  et  al.,  1978) 
depict  a buildup  of  forces  of  the  defending  German  Army  that  eventually 
culminated  in  an  aggressive,  large-scale  counteroffensive  and  break- 
through of  the  Allied  lines  on  16  December,  known  as  the  Battle  of  the 
Bulge.  After  the  fact,  it  is  possible  to  deduce  the  significance  of  the 
intelligence  gathered  at  the  time. 

I i 
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It  is  known,  however,  that  the  strength  of  the  attack  by  a presum- 
ably defeated,  demoralized,  and  hard-pressed  enemy  caught  the  Allies  by 
surprise.  The  messages  range  from  trivial  sighting  reports  of  horse- 
drawn  carts  to  reports  of  massive  troop  movements,  POW  interrogations, 
and  G2  summaries.  They  vary  in  length,  have  different  degrees  of  com- 
pleteness, and  originate  from  several  agencies  and  sources.  Although 
the  situation  is  dated  and  involves  a style  of  warfare  that  differs 
sharply  from  more  recent  conflicts,  these  messages,  unlike  fictional 
scenarios,  are  real,  have  a documented  outcome,  and  are  natural istically 
valid. 

The  second  scenario  was  placed  in  the  context  of  a hypothetical 
engagement  in  the  Hof  Gap  area  of  modern-day  central  Europe.  Forty 
messages  (see  Appendix  A)  were  taken  from  a scenario  developed  to  study 
intelligence  processing  in  an  Order-of-Battle  (OB)  section  (Halpin, 
Staniforth,  and  Bowen,  1973) . The  messages  were  selected  from  the 
large  number  that  would  be  received  by  the  OB  section  during  one  12-hour 
shift  and  reflect  the  activities  of  an  Aggressor  Combined  Arms  Army  as 
it  moves  from  a training/maneuver  location  and  begins  to  deploy  for  an 
attack  to  the  west.  The  messages  include  reports  of  troop  movements, 
rumors  of  enemy  plans,  enemy  air  and  radio  activities,  and  so  forth. 

They  roughly  match  those  in  the  Battle  of  the  Bulge  scenario  in  terms 
of  source  (e.g.,  proportion  of  reports  from  friendly  civilians  versus 
prisoners  of  war  (POW) ) and  routing  (e.g.,  messages  from  higher,  lower, 
or  adjacent  commands)  of  the  information. 


Procedure 


Each  group  of  participants  was  assembled  in  a large  classroom  and 
was  given  a briefing  on  the  purpose  of  the  study,  the  participants ' 
tasks,  and  the  scenario  they  would  be  dealing  with.  For  participants 
dealing  with  the  well-known  Battle  of  the  Bulge,  the  situation  briefing 
focused  on  the  current  military  aspects,  including  locations  of  friendly 
and  enemy  forces.  There  was  little  discussion  of  earlier  engagements, 
German  and  Allied  strategy,  etc.  The  Hof  Gap  participants  received 
somewhat  more  background  on  the  political  situation,  recent  military 
activity,  and  indications  of  enemy  intentions.  Following  the  introduc- 
tory briefing,  each  participant  was  given  a packet  of  materials,  includ- 
ing, instructions  for  rating  the  messages;  response  sheets;  messages  to 
be  rated;  a map  sheet  of  the  area  of  interest  and  an  overlay  showing  the 
current  tactical  situation;  and  grease  pencils  for  plotting  information 
received. 


Task 


Participants  were  asked  to  read  the  background  materials  provided 
and  examine  available  maps  and  overlays.  Then  individuals  evaluated  the 
<juality  of  each  of  the  40  messages.  Evaluations  were  made  by  rating  each 
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message  on  each  of  the  50  scales.  As  in  the  Miron  et  al.  (1978)  study, 
participants  were  given  a set  of  rating  forms  (Figure  1)  containing  the 
46  bipolar  adjective  scales,  the  standard  Accuracy  and  Reliability  scales, 
and  the  Global  Validity  scales.  After  they  had  provided  these  50  rat- 
ings on  all  40  messages,  they  were  given  a representative  sample  of  20 
messages  chosen  from  the  40  they  had  already  rated  and  were  asked  to  rate 
each  message  on  the  four  new  rating  scales  developed  from  the  findings  of 
the  Miron  et  al.  (1978)  study  (Figure  2). 

The  task  required  approximately  4 hours  to  complete.  Two  monitors 
were  present  during  each  of  the  experimental  sessions  to  provide  instruc- 
tions, answer  questions,  and  check  on  progress  for  each  group. 


RESULTS 

The  basic  data  consist  of  more  than  120,000  observations — the 
ratings  by  each  of  the  28  subjects  in  the  two  groups  on  each  of  the  50 
scales  for  each  of  the  40  messages  in  a scenario  plus  each  of  4 scales 
for  20  of  the  messages.  The  data  may  be  described  in  terms  of  two 
three-dimensional  matrices,  one  for  each  of  the  two  scenarios,  where  the 
dimensions  identify  messages,  scales,  and  subjects.  The  analyses  related 
to  the  50  general  scales  are  presented  for  each  of  the  two  scenarios 
prior  to  presenting  analyses  for  the  four  new  scales. 


Overview  of  the  Analysis 

Our  concern  is  with  the  relationship  among  the  scales  rather  than 
differences  among  messages  or  participants.  Therefore,  the  first  step 
was  to  find  the  average  (mean)  rating  made  on  a given  scale  for  a given 
message  across  all  subjects  for  each  scenario  (Table  2) . Then  the 
correlations  among  the  scales  across  the  messages  were  computed.  The 
resulting  scales  intercorrelation  matrix  was  subjected  to  a Principal 
Components  Factor  Analysis  with  Varimax  rotation,  a technique  that  leads 
to  a description  of  the  underlying  relationships  among  the  scales.  The 
basic  assumption  underlying  the  use  of  this  analysis  in  this  context  is 
that  an  individual's  "concept  space"  or  "judgment  space"  may  be  described 
by  a relatively  few  independent  dimensions  in  the  same  way  that  physical 
space  may  be  described  in  terms  of  "height, * "width,"  and  "depth."  The 
factor  analysis  of  the  scales  intercorrelation  matrix  may  be  thought  of 
as  identifying  rating  scales  that  were  used  in  a similar  fashion;  the 
experimenter  uses  these  groups  of  similar  scales  to  define  the  dimensions 
of  the  judgment  space. 

The  Principal  Components  technique  operates  by  successively  extract- 
ing dimensions  or  factors  that  account  for  the  maximum  total  variance  in 
the  matrix  of  interest,  taking  into  consideration  the  variance  already 
accounted  for  by  earlier  dimensions.  The  Varimax  rotation  has  the 
effect  of  relocating  the  dimensional  structure  to  spread  the  factor 
variance  more  evenly  across  the  separate  factors.  Each  factor  after 
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Table  2 

Means  and  Standard  Deviations  for  Each  Rating  Scale 
Across  All  Messages  and  Raters 


Scenario 


Scale 

Battle  of  the  Bulge 

Hof  Gap 

X 

s .d. 

X 

s.d. 

1. 

Active/Inactive 

3.08 

1.50 

3.08 

1.21 

2. 

False/True 

5.30 

1.25 

5.03 

1.19 

3. 

Untimely/Timely 

5.20 

1.58 

5.08 

1.33 

4. 

Spec i f i c/Vague 

3.41 

1.72 

3.55 

1.53 

5. 

Precar ious/Imprecarious 

3.62 

1.29 

3.75 

1.06 

6. 

Extraordinary /Ordinary 

3.95 

1.50 

3.91 

1.22 

7. 

Intelligible/Unintelligible 

2.78 

1.34 

2.98 

1.18 

8. 

Infeasible/Feasible 

5.64 

1.24 

5.41 

1.23 

9. 

Factual/Theoretical 

3.02 

1.49 

3.17 

1.19 

10. 

Unstable/Stable 

4.36 

1.31 

4.19 

1.17 

11. 

Direct/Indirect 

3.17 

1.47 

3.37 

1.23 

12. 

Variable/Reliable 

4.95 

1.42 

4.70 

1.29 

13. 

Understandable/Confusing 

2.88 

1.48 

3.04 

1.35 

14. 

Likely /Unlikely 

2.77 

1.39 

2.88 

1.30 

15. 

Massive/Insignificant 

3.43 

1.46 

3.46 

1.05 

16. 

Contradicted/Substantiated 

4.60 

1.16 

4.40 

1.02 

17. 

Dependable/Undependable 

3.01 

1.37 

3.29 

1.22 

18. 

Useless/Useful 

5.18 

1.44 

5.15 

1.19 

19. 

Inert/Volatile 

4.46 

1.33 

4.53 

1.08 

20. 

Inappropriate/Appropriate 

5.12 

1.33 

5.10 

1.21 

21. 

Truthful/Deceptive 

3.00 

1.36 

3.26 

1.23 

22. 

Unhazardous /Hazardous 

4.47 

1.18 

4.44 

.98 

23. 

Expected/Unexpected 

3.78 

1.39 

3.45 

1.41 

24. 

Unimplied/Implied 

3.97 

1.37 

4.00 

1.23 

25. 

Heavy /Light 

3.65 

1.22 

3.76 

.83 

26. 

Possible/ Impossible 

2.42 

1.17 

2.64 

1.12 

27. 

Observed/Inferred 

3.18 

1.67 

3.48 

1.39 

28. 

Garbled/Clear 

4.99 

1.28 

4.77 

1.13 

29. 

Faultless/Faulty 

3.67 

1.21 

3.84 

1.08 

30. 

Widespread/Local 

3.88 

1.46 

4.07 

1.24 

31. 

Accurate/Erroneous 

3.09 

1.26 

3.28 

1.15 

32. 

Uninterpreted/Interpreted 

3.94 

1.52 

3.96 

1.37 

33. 

Relevant/Irrelevant 

2.69 

1.32 

2.68 

1.16 

34. 

Insecure/Secure 

4.08 

1.03 

4.16 

.79 

35. 

Analyzable/Unanalyzable 

2.85 

1.34 

3.11 

1.19 

36. 

Risky/Routine 

3.80 

1.25 

3.86 

.96 

37. 

Fragmented/ Completed 

4.14 

1.39 

3.91 

1.07 

1 


Table  2 (Continued) 


Scenario 


Scale 

Battle  of 

the  Bulge 

Hof 

Gap 

X 

s.d. 

X 

s.d. 

38. 

Acceptable/Unacceptable 

3.08 

1.39 

3.24 

1.28 

39. 

Many/Few 

3.79 

1.32 

3.92 

.72 

40. 

Confinned/Unconfirmed 

3.82 

1.59 

4.14 

1.51 

41. 

Changing/Constant 

3.52 

1.22 

3.79 

1.07 

42. 

Large  Scale/Small  Scale 

3.55 

1.48 

3.78 

1.07 

43. 

Improbable/Probable 

5.24 

1.35 

5.13 

1.29 

44. 

Extraneous/Pertinent 

5.11 

1.41 

5.25 

1.22 

45. 

Consistent/Inconsistent 

3.24 

1.38 

3.26 

1.35 

46. 

Dangerous/Safe 

3.58 

1.19 

3.63 

.89 

47. 

Reliability 

3.05 

1.71 

3.67 

1.88 

48. 

Accuracy 

3.02 

1.48 

3.22 

1.59 

49. 

Global  I 

73.99 

21.94 

72.10 

22.66 

50. 

Global  II 

73.37 

22.67 

71.99 

23.66 

rotation  is  perpendicular  to  all  other  factors;  thus,  variation  within 
one  factor  is  independent  of  variation  in  every  other  factor.  (For  a 
more  detailed  discussion  of  factor  analysis,  the  underlying  assumptions, 
the  mathematical  operations,  and  interpretation  and  application  of  the 
results,  see  Harman,  1967.) 

The  results  of  a factor  analysis  are  typically  presented  in  terms 
of  the  "loadings"  of  each  of  the  items,  in  this  case  rating  scales,  on 
each  of  the  factors.  The  absolute  magnitude  of  the  'loading  of  a 
particular  scale  on  a factor  reflects  the  relative  importance  of  that 
scale  as  a representative  of  the  underlying  judgment  dimension  being 
used  by  the  subjects.  All  scales  have  loadings  on  all  factors,  and  any 
listing  of  scales  that  load  "heavily"  on  a factor  involves  a somewhat 
arbitrary  cutoff  at  the  point  where  it  appears  that  including  any 
additional  scales  would  contribute  little  to  an  understanding  of  the 
underlying  dimension  of  judgment.  Similarly,  the  choice  of  the  number 
of  factors  to  display  and  discuss  involves  a judgment  that  the  additional 
information  provided  by  the  successively  less  powerful  factors  provides 
little  additional  understanding  of  the  judgments  underlying  the  factor 
space. 

As  shown  in  Figure  1,  an  attempt  was  made  to  design  the  rating  form 
to  avoid  problems  of  response  bias.  It  is  well  known  that  some  indi- 
viduals have  a tendency  to  respond  on  the  "low"  or  left-hand  end  of  rating 
scales  while  others  have  a similar  tendency  to  respond  on  the  "high"  or 
right-hand  end  of  such  scales.  The  orientation  of  scales  relating  to 
similar  concepts  was  counterbalanced,  e.g.,  False/True,  Likely/Unlikely, 
Truthful/Deceptive,  and  Improbable/Probable,  to  minimize  the  influence 
of  such  response  biases.  The  result  of  this  counterbalancing  procedure 
is  a number  of  negative  correlations  among  related  concepts;  a report 
rated  closer  to  the  True  end  of  the  False/True  scale  (and  hence  assigned 
a high  number)  would  also  be  rated  close  to  the  Truthful  end  of  the 
Truthful/Deceptive  scale  (and  hence  assigned  a low  number) . 

To  simplify  the  presentation  of  the  results  by  minimizing  the 
number  of  negative  relationships , the  data  from  the  scales  are  reported 
in  most  instances  as  though  there  had  been  a common  orientation.  The 
exceptions  are  the  traditional  Accuracy  and  Reliability  scales,  which 
are  consistently  presented  with  "1"  as  the  "high"  end  (Confirmed  by 
Other  Sources,  Completely  Reliable),  and  the  100-point  scales,  which 
are  scored  with  "0"  as  the  low  end  and  "100"  as  the  high  end. 

When  comparing  "similar"  factors  from  different  analyses,  it  should 
be  noted  that  the  appearance  of  an  identical  set  of  rating  scales  in 
identical  order  would  be  very  unlikely.  If  a common  judgment  dimension 
is  present,  however,  then  scales  with  a similar  meaning  will  have  similar 
loadings.  To  provide  some  "objective"  basis  for  the  comparison  of  the 
factors  obtained  in  the  study  with  those  previously  found  by  Miron  et  al., 
(1978) , correlations  were  computed  between  the  factor  loadings  on  factors 
obtained  from  the  samples.  These  correlations  must  be  interpreted  very 


cautiously,  because  no  basis  exists  for  assessing  or  establishing  the 
statistical  significance  of  these  coefficients.  Furthermore,  because 
the  correlations  are  computed  on  the  basis  of  all  factor  loadings  and 
not  just  the  "meaningful"  loadings,  the  correlations  may  be  unduly 
influenced  by  essentially  random  variation  in  the  low  loadings.  Thus, 
these  correlations  must  be  considered  as  indicative  rather  than  as  firm 
evidence  of  relationships. 


In  the  presentation  and  the  discussion  of  the  results,  there  is 
some  potential  for  confusion  due  to  the  fact  that  the  labels  assigned 
to  the  factors  found  in  different  experiments  may  be  similar  or  identical. 
To  minimize  confusion,  factor  labels  will  include  a superscript  to 
indicate  the  experiment  from  which  the  data  were  derived.  Factor  labels 
from  the  Miron  et  al.  (1978)  study  will  have  a "0"  superscript,  those 
based  on  the  Battle  of  the  Bulge  senario  will  have  a "1"  superscript, 
and  those  based  on  the  Hof  Gap  scenario  will  have  a "2"  superscript. 

Thus,  TRUTH®  and  TRUTH 2 would  refer  to  factors  derived  from  data  from 
the  Miron  et  al.  (1978)  study  and  from  the  Hof  Gap  scenario,  respectively. 
The  use  of  labels  without  a superscript  will  refer  to  a generalized 
concept  based  on  an  integration  of  results  from  more  than  one  set  of 
data. 


Battle  of  the  Bulge  Scenario 


The  results  of  the  factor  analysis  based  on  the  ratings  made  by 
the  28  participants  in  the  Battle  of  the  Bulge  scenario  are  presented  in 
Table  3.  The  four  rotated  factors  account  for  84.4%  of  the  variance 
among  messages  in  mean  scale  ratings.  Factor  I,  which  accounts  for 
48.8%  of  the  variance,  has  high  loadings  for  scales  such  as  Probable/ 
Improbable,  Likely /Unlikely , Global  Validity,  and  other  scales  that 
presumably  reflect  the  judged  likelihood  that  the  information  would  be 
received  in  the  context  of  the  scenario  and  the  judged  truth  of  the 
information.  This  factor  is  called  TRUTH-*-. 


Factor  II  reflects  scales  such  as  Routine/Risky,  Safe/Dangerous, 
and  Small  Scale/Large  Scale,  which  deal  with  the  potential  threat  and 
the  scope  (size  and  amount)  of  activity  reported  in  the  messages.  This 
factor,  which  accounts  for  25.9%  of  the  variance,  is  labeled  IMPORTANCE1. 
Factor  III  accounts  for  only  5.9%  of  the  variance  and  includes  both  of 
the  standard  scales.  Accuracy  and  Reliability.  But  the  other  scales 
loading  heavily  here,  such  as- Dependable/Undependable , indicate  that 
the  underlying  judgment  is  of  SOURCE  RELIABILITY1.  Factor  IV,  labeled 
AMBIGUITY1,  accounts  for  only  4.2%  of  the  variance. 


The  correlations  between  the  factors  obtained  in  the  present 
experiment  and  the  corresponding  factors  from  the  analysis  of  the 
combined  sample  in  the  Miron  et  al.  (1978)  study  are  shown  in  Table  4. 


Rating  Factors  for  Analysts  Using  the  Battle  of  the  Bulge  Scenario 
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There  is  a strong  similarity  between  the  Miron  et  al.  results  and 
the  results  obtained  in  the  current  experiment.  In  particular,  the 
first  two  factors  in  each  experiment  are  clearly  related  to  judgments 
of  TRUTH  and  IMPORTANCE,  respectively.  The  major  difference  between  the 
two  results  is  that  the  relatively  inexperienced  enlisted  analysts 
apparently  did  not  differentiate  between  information  accuracy  and  the 
reliability  of  the  source.  The  current  sample  of  officers,  all  with 
several  years  of  experience,  did  make  an  independent  judgment  of  Source 
Reliability.  It  is  interesting  to  note  that  these  officers  used  the 
traditional  Accuracy  scale  in  a way  that  shows  this  judgment  to  be  tied 
closely  to  their  judgment  of  Source  Reliability  rather  than  to  their 
judgment  of  the  TRUTH  of  the  information. 


Hof  Gap  Scenario 

The  results  of  the  factor  analysis  of  the  mean  ratings  of  the  28 
participants  who  evaluated  these  40  messages  are  presented  in  Table  5. 
The  four  rotated  factors  account  for  82.3%  of  the  variance  in  the 
averaged  scale  ratings  of  the  messages.  Factor  I,  which  accounts  for 
53.9%  of  the  variance,  has  high  loadings  for  scales  such  as  Accurate/ 
Erroneous,  False/True,  Truthful/Deceptive,  Improbable/Probable,  and 
Possible/Impossible.  This  factor  is  labeled  TRUTH2.  Factor  II  reflects 
the  use  by  analysts  of  scales  such  as  Precarious/Imprecarious  and 
Dangerous /Safe,  which  reflect  the  possible  impact  of  the  activity 
reported  in  the  messages  on  the  analyst's  unit.  This  factor,  which 
is  labeled  THREAT2,  accounts  for  15.9%  of  the  variance.  Factor  III 
accounts  for  only  7.4%  of  the  variance;  this  factor  apparently  reflects 
a judgment  of  the  SCOPE2  of  activity  reported — Large  Scale/Small  Scale, 
Widespread/Local,  Many /Few.  Factor  IV,  tentatively  labeled  CLARITY2, 
accounts  for  only  5.1%  of  the  variance. 

The  correlations  between  the  factors  obtained  in  the  Hof  Gap 
scenario  and  the  Battle  of  the  Bulge  scenario  as  well  as  with  the 
Miron  et  al.  (1978)  results  are  shown  in  Table  4. 

A comparison  of  the  Hof  Gap  scenario  results  with  the  Battle  of  the 
Bulge  results  shows  a overall  similarity  of  the  judgment  dimensions 
used  by  these  two  samples  of  officers  in  evaluating  different  sets  of 
combat  intelligence  messages.  As  in  the  Battle  of  the  Bulge  scenario, 
the  principal  judgment  is  of  the  TRUTH  of  the  information.'  In  the  Hof 
Gap  scenario,  however,  we  find  the  analysts  combining  the  notions  of 
Accuracy  and  Source  Reliability  in  their  judgment  of  TRUTH2,  whereas 
the  analysts  in  the  Battle  of  the  Bulge  scenario  used  Accuracy  and 
Source  Reliability  as  components  of  a separate  factor,  SOURCE 
RELIABILITY1. 

The  present  results  also  reflect  the  use  of  judgments  concerning 
the  THREAT2  and  SCOPE2  of  information;  here  these  judgments  appear  as 
independent  factors,  whereas  before  they  had  been  combined  into  one 


Rating  Factors  for  Analysts  Using  the  Hof  Gap  Scenario 


factor,  IMPORTANCE^.  As  in  the  previous  results,  there  is  a weak  but 
consistent  judgment  of  message  AMBIGUITY  or  CLARITY. 


Candidate  Scales 

Factor  analysis  was  used  to  relate  the  four  candidate  scales — 
"True,"  "Relevant,"  "Predictable,"  and  "Important" — to  the  underlying 
dimensions  of  judgment.  Evaluation^  from  the  four  scales  were  included 
with  those  from  the  50  general  scales  in  an  analysis  of  the  20  messages 
that  had  been  selected  from  each  scenario. 

The  results  of  the  factor  analysis  based  on  the  mean  ratings  across 
the  28  officers  of  the  Battle  of  the  Bulge  messages  are  presented  in 
Table  6,  and  the  results  based  on  the  mean  ratings  across  the  second 
group  of  28  officers  of  the  central  European  scenario  are  presented  in 
Table  7.  As  expected,  the  basic  factor  structure  is  similar  to  that 
found  for  the  complete  sets  of  messages.  Of  more  interest  here  is  the 
location  of  the  four  new  scales  in  the  previously  established 
dimensional  structure. 

Both  groups  of  analysts  used  the  new  0-100  "True"  scale  in  such  a 
way  that  the  scale  clearly  represents  the  TRUTH  dimension  underlying  the 
first  factor  obtained  in  the  factor  analysis  solutions.  In  addition, 
the  new  Predictable  scale  is  weakly  identified  with  the  TRUTH  dimension 
in  both  analyses.  However,  as  may  be  seen  in  Table  6,  the  analysts 
rating  the  Battle  of  the  Bulge  messages  also  used  the  Predictable  scale 
in  the  context  of  a fourth  factor  which  apparently  reflects  a judgment 
of  the  "expectedness"  of  information. 

For  the  analysts  evaluating  the  Battle  of  the  Bulge  messages, 
the  new  0-100  "Important"  and  "Relevant"  scales  are  both  identified 
with  Factor  II,  which  reflects  an  IMPORTANCE!  dimension.  The  analysts 
evaluating  the  Hof  Gap  messages,  however,  did  not  make  their  ratings  in 
terms  of  importance,  but  rather  made  independent  judgments  on  DANGER2 
and  SCOPE2  dimensions.  For  these  analysts,  both  the  new  Important  and 
Relevant  scales  are  weakly  identified  with  the  DANGER2  dimension. 


DISCUSSION 

The  results  of  the  present  research  suggest  that  there  is  a basic 
commonality  in  the  conceptual  or  judgment  structure  used  by  different 
intelligence  analysts  to  evaluate  combat  intelligence  data.  The  high 
degree  of  similarity  between  the  results  obtained  from  three  samples  of 
analysts  (untrained  and  school-trained  enlisted  intelligence  analysts 
and  company  grade  intelligence  officers)  and  from  two  different 
scenarios  supports  the  existence  of  judgment  dimensions  that  are  basic 
to  the  evaluation  of  combat  intelligence  data. 
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only  a weak  relationship  to  the  factor  but  is  included  to  show  how  it  fits  within 


Evaluation  of  the  quality  of  combat  intelligence  data  appears  to 
be  based  on  four  independent  factors.  Factor  I,  TRUTH,  accounts  for 
about  50%  of  the  variance  in  message  ratings  in  both  scenarios  and  in 
all  three  groups  of  analysts.  Factor  II,  RELEVANCE/IMPORTANCE/THREAT, 
accounts  for  about  20%  of  the  variance  in  message  ratings  in  both 
scenarios  and  in  all  three  groups  of  analysts.  Factors  III  and  IV 
account  for  about  10%  of  the  variance  in  message  ratings,  but  they  are 
not  stable  and  vary  somewhat  across  scenarios  and  across  analysts. 
Considerations  of  TRUTH  and  IMPORTANCE  are  clearly  the  critical  factors 
in  the  evaluation  of  intelligence  data. 

The  primary  factor  of  message  quality  is  TRUTH.  The  current 
Reliability  and  Accuracy  ratings  are  generally,  but  not  always,  compo- 
nents of  the  TRUTH  factor.  In  one  exception,  intelligence  officers  using 
the  Hof  Gap  scenario  had  both  Reliability  and  Accuracy  as  components  of 
a third  factor,  source  RELIABILITY.  In  all  instances,  the  TRUTH  aspect 
of  message  quality  is  related  to  such  bipolar  scales  as  Probable/Improb- 
able, True/False,  Acceptable/Unacceptable,  and  the  Global  Validity  scales. 
A second  significant  factor  of  message  quality  is  IMPORTANCE,  which  is 
described  by  such  bipolar  scales  as  Routine/Risky,  Safe/Dangerous, 
Massive/Insignificant,  and  Volatile/Inert . 

Although  the  judgment  space  was  nearly  identical  for  the  three 
' groups  of  analysts  as  well  as  for  the  two  scenarios,  individuals  did 
not  necessarily  evaluate  the  reports  identically.  Similarities  exist 
in  the  structured  space  or  dimensions  within  which  information  is 
evaluated.  For  example,  different  analysts  will  consider  as  critical 
the  same  information  qualities  we  have  identified  as  TRUTH  and 
IMPORTANCE  for  all  messages.  However,  evaluations  of  the  quality  of 
specific  messages  by  different  analysts  are  likely  to  be  different. 

The  situation  is  analogous  to  different  observers  who  can  see  the  same 
visual  stimulus,  i.e.,  respond  to  the  same  energy  inputs,  but  who  have 
different  views  on  the  significance  or  interpretation  of  that  stimulus. 

The  significance  of  the  findings  concerning  the  dimensions  of 
judgment  used  to  evaluate  combat  intelligence  data  is  threefold. 

First,  the  analyst  brings  to  his  task  an  implicit  judgment  structure 
concerning  information  quality.  Second,  the  implicit  judgment  structure 
of  the  analyst  does  not  correspond  to  the  evaluation  structure  embodied 
in  current  doctrine.  Third,  this  implicit  judgment  structure  does  not 
appear  to  be  modified  by  the  training  procedures  employed  at  the  U.S. 

Army  Intelligence  Center  and  School.  Thus,  the  judgment  structure  of 
company  grade  intelligence  officers  is  similar  to  the  judgment  structure 
of  prospective  enlisted  intelligence  analysts  just  entering  training. 

What  we  know  to  be  true  of  such  implicit  judgment  structures  indicates 
that  it  would  be  very  difficult  to  modify  them. 

The  present  structure  for  evaluating  the  quality  of  intelligence 
data  is  based  on  four  dimensions  (FM  30-5,  1973):  Accuracy  of 
Information,  Reliability  of  Source,  Timeliness,  and  Pertinence.  These 
four  factors. do  not  have  a close  relationship  to  TRUTH  and  IMPORTANCE, 
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the  primary  factors  on  which  analysts  appear  to  base  their  judgments 
of  information  quality.  For  example,  TRUTH  includes  both  the  aspects 
of  information  quality  evaluated  by  the  current  ratings.  Accuracy  of 
Information  and  Reliability  of  Source,  as  well  as  aspects  related  to 
scales  such  as  Probable/Improbable  and  Consistent/Inconsistent.  Thus, 
many  of  the  deficiencies  of  the  current  evaluation  structure  may  arise 
from  the  mismatch  between  the  implicit  judgment  structure  of  analysts 
and  the  evaluation  structure  embodied  "in  current  doctrine.  What  is 
required  is  that  doctrine  and  training  use  the  underlying  ccnceptual 
structure  of  information  quality  to  best  advantage. 

The  four  candidate  scales  results  obtained  that  indicate  it  is 
possible  to  develop  simple,  numerical  scales  that  represent  the  factors 
of  this  conceptual  judgment  structure.  This  is  most  clearly  seen  in 
the  case  of  the  new  0-100  "True"  scale,  which  was  found  to  be  an 
important  element  of  a TRUTH  factor.  However,  the  remaining  three 
scales.  Relevant,  Predictable,  and  Important,  were  not  as  successful 
in  capturing  other  factors  in  the  analysts'  judgment  structure.  The 
Predictable  scale  was  both  highly  correlated  with  the  True  scale  and 
an  element  of  the  TRUTH  factor.  The  Important  and  Relevant  scales 
were  highly  intercorrelated  and  both  were  components  of  IMPORTANCE. 

The  data  show  that  a "True"  scale  provides  a basis  for  the  development 
of  evaluation  procedures  of  a TRUTH  factor,  but  further  exploration 
of  various  scales  appears  necessary  to  begin  development  of  improved 
procedures  for  evaluating  other  factors  in  the  judgment  structure  of 
analysts . 

The  results  of  the  four  candidate  scales  also  indicate  that  a 
single  key  word  or  word  pair  will  not  clearly  capture  the  underlying 
concept  represented  by  any  of  the  judgment  dimensions.  For  example, 
requiring  analysts  to  provide  a rating  of  a message's  truth,  accuracy, 
or  validity  will  not  necessarily  yield  consistent,  reliable  judgments 
from  all  analysts  for  all  tactical  situations  because  the  descriptive 
word  chosen  may  not  represent  the  analyst's  personal  concept  of 
TRUTH.  Nevertheless,  it  is  clear  that  a great  deal  of  information 
about  data  quality  can  be  conveyed  by  a numerical  rating  on  a "True" 
scale  that  is  interrelated  with  other  aspects  of  the  general  concept 
of  TRUTH.  Numerical  rating  scales  with  equal  intervals  should  be 
used  instead  of  scales  such  as  the  present  Information-Accuracy /Source 
Reliability  scales,  which  depend  on  verbal  labels  and  their  specific 
definitions.  Further  research  may  be  necessary  to  determine  the  optimal 
scale  form  (e.g.,  0-100;  0-10;  etc.)  and  if  such  a rating  will  provide 
sufficient  information  about  the  data  for  proper  use  by  a recipient. 

The  TRUTH  factor  accounts  for  about  50%  of  the  variability  in 
information  quality,  and,  for  many  purposes,  a Truth  or  Accuracy 
scale  would  provide  intelligence  and  command  personnel  with  a sufficient 
basis  for  information  utilization.  This  scale  includes  the  aspects  of 
information  quality  evaluated  by  the  present  Information  Accuracy  and 
Source  Reliability  ratings  and  could  easily  be  incorporated  into 
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existing  doctrine.  The  suggestion  is  to  develop  a truth  scale  whose 
label  is  related  to  the  current  ratings,  i.e.,  information  accuracy. 
However,  this  scale  would  represent  a very  different  and  more  natural 
rating  scheme  than  the  existing  system. 

Although  the  new  Accuracy  scale  appears  to  capture  a major  portion 
of  the  variance  in  evaluations  of  information  quality,  the  two 
experiments  supporting  this  conclusion  have  been  conducted  in  a 
constrained  laboratory  environment.  The  new  Accuracy  scale  should  be 
evaluated  in  the  field  for  user  acceptance  and  for  validation  of  the 
findings.  A critically  important  aspect  of  validation  is  to  determine 
if  this  new  accuracy  scale  would  satisfy  the  requirements  of  different 
classes  of  users. 

Current  doctrine  is  based  on  the  implicit  assumption  that  the 
rating  of  information  quality  serves  several,  not  necessarily  compatible 
purposes.  For  example,  ratings  of  the  quality  of  information  are 
used  by  analysts  to  integrate  effectively  information  from  several 
sources;  collection  managers  use  ratings  to  develop  data  necessary  for 
effective  resource  allocation  and  to  provide  feedback  to  information 
sources  on  the  quality  of  their  product.  A numerical  rating  of 
information  "Accuracy"  should  satisfy  the  needs  of  different  categories 
of  users.  However,  this  is  an  empirical  issue  and  should  be  resolved 
through  laboratory  or  field  research. 

A final  problem  to  be  considered  is  that,  under  some  circumstances, 
this  new  Accuracy  Scale  may  not  give  a sufficiently  comprehensive 
evaluation  of  information.  Although  Accuracy  is  a primary  rating 
quality,  it  may  be  supplemented  with  other  ratings,  e.g.,  an  "Importance 
scale  representing  IMPORTANCE/RELEVANCE/THREAT.  The  present  research 
suggests  that  there  are  circumstances  when  several  numerical  rating 
scales  may  be  needed  to  allow  the  analyst  to  communicate  evaluation 
of  the  information  on  each  of  the  possibly  relevant  dimensions  of  data 
quality. 


CONCLUSIONS  AND  RECOMMENDATIONS 

Evaluations  of  the  quality  of  combat  intelligence  data  appear  to 
be  based  on  four  independent  factors.  The  primary  factor  of  message 
quality  is  TRUTH,  which  has  as  its  components  the  current  Reliability 
and  Accuracy  ratings.  A second  significant  factor  is  RELEVANCE/ 
IMPORTANCE/THREAT.  Two  additional  significant,  but  much  less  powerful, 
factors  are  not  clearly  identified  and  tend  to  vary  between  different 
types  of  intelligence  personnel  and  different  scenarios. 

This  conceptual  structure  is  not  well  defined  by  the  current 
information  Accuracy  and  Source  Reliability  ratings.  A new  scale  has 
been  developed  to  represent  the  TRUTH  factor  of  the  conceptual  structure 


The  scale  for  TRUTH  should  be  evaluated  in  the  field  and,  if  the  present 
results  are  validated,  then  it  should  be  considered  as  a replacement  for 
the  current  rating  scales.  It  is  evident  from  the  present  research  that 
additional  research  is  required  to  develop  rating  scales  for  the  second 
factor , RELEVANCE/IMPORTANCE/THREAT . 
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APPENDIX 


CENTRAL  EUROPEAN  SCENARIO 


20TH  Mech  Division 


Selected  Messages 
121800  - 130500  SEP 


Alignment  of  Units  in  30th  (US  Army 
from  North  to  South) 


2nd  (US)  Corps 


3rd  (US)  Corps 
57th  Mech  Division 
20th  Mech  Division 
74th  Mech  Division 
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MSG  NO.  1 


FROM:.  S-2,  1BDE,  121811  SEP 
TO:  G-2 , 20  MECH  DIV 

UNIT  IDENTIFIED  AS  2ND  BN,  213  MECH  REGT,  HAS  BEEN  REPORTED  BY 
FRIENDLY  CIVILIANS  TO  BE  LOCATED  IN  THE  VICINITY  OF  NEUSTADT 
(UR  1092)  ACCORDING  TO  THESE  REPORTS  THE  UNIT  STRENGTH  IS 
APPROXIMATELY  83%  OF  TO&E. 

MSG  NO.  2 

FROM:  S-2,  2BDE,  121829  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIAN  HAS  REPORTED  THAT  HE  WAS  PART  OF  A WORK  FORCE  THAT  ASSISTED 
IN  THE  EMPLACEMENT  OF  AGGRESSOR  MINE  FIELDS  ON  10/11  SEP.  THE  FIELDS 
ARE  PLACED  IN  RECTANGULAR  PATTERNS  IN  UNSPECIFIED  AREAS  FROM  3-7 
KILOMETERS  TO  THE  FRONT  OF  20  MECH  DIV  LINE  OF  CONTACT.  EMPLACEMENT 
WAS  SUPERVISED  BY  ENGINEER  OFFICERS  THAT  GAVE  THE  INDICATION  THAT  THEY 
WERE  FROM  16  CAA  STAFF. 

MSG  NO.  3 

FROM:  G-2,  AIR,  III  CORPS,  121837  SEP 
TO:  G-2,  20  MECH  DIV 

AGGRESSOR  AERIAL  RECONNAISSANCE  HAS  INCREASED  ALONG  THE  III  CORPS 

FRONT  10-12  SEP.  AVAILABLE  DATA  INDICATES  MISSIONS  ARE  SLAR, 

INFRARED,  AND  OBLIQUE  PHOTOGRAPHY.  CARE  WAS  TAKEN  BY  AGGRESSOR 

AIRCRAFT  NOT  TO  CORSS  THE  BORDER. 

MSG  NO.  4 

FROM:  G-2,  57  MECH  DIV,  121841  SEP 
TO:  G-2,  20  MECH  DIV 

AGENT  REPORTS  A LARGE  VOLUME  OF  VEHICULAR  TP.AFFIC  BOTH  INCOMING  AND 

OUTGOING,  IN  THE  VICINITY  OF  LENGENFELD  (US  1305)  120830-121640  SEP. 

APPARENT  CARGO  WAS  POL,  SUPPLIES,  AND  TROOP  REPLACEMENTS. 
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MSG  NO.  5 

FROM:  S-2 , 1BDE,  121851  SEP 
TO:  G-2 , 20  MECH  DIV 

AGENT  REPORTS  INDICATE  THE  POSSIBLE  PRESENCE  OF  THE  54  MECH  DIV, 

19  CAA  IN  THE  VICINITY  OF  AUERBACH  (UR  1699).  INFORMATION  WAS 
OBTAINED  FROM  OVERHEARD  CONVERSATIONS  AND  SIGHTINGS  OF  VEHCILES. 

MSG  NO.  6 

FROM:  IPW  SECTION,  20  MECH  DIV,  121933  SEP 
TO:  G-2,  20  MECH  DIV 

SUBJECT : DESERTER 
NAME:  KNORR,  ABEL  R. 

RANK:  SGT 

SERIAL  NUMBER:  296361 

UNIT:  1ST  BN,  IF  MDM  TK  REGT,  IF  MECH  DIV 

DESERTER  PICKED  UP  BY  PATROL  AT  121526  VIC  TR  973862.  STATES  HIS 
UNIT  WAS  LOCATED  UR  037847.  BN  CDP  IS  PRESNOFF,  IVAN  R.  AND  REGT  CDR 
IS  COL  WEIR,  V.  DESERTER  FURTHER  STATES  HIS  FATHER  (KNORR,  JOST,  COL,  RFL . ) 
COMMANDED  A REGT  IN  HIS  DIV.  (211  MECH  REGT)  DESERTED  BECAUSE  OF  THE 
DIFFICULT  TIME  HE  HAD  IN  KEEPING  WITH  HIS  FATHER'S  IMAGE  OF  A SOLDIER. 


MSG  NO.  7 

FROM:  DIV  AIRFLD  TIFF,  122036  SEP 

TO:  G-2,  20  MECH  DIV 

FLIGHT  NO.  010 

MSN  NO.  SLAR 

RANGE:  10/30 

ALT  5000  FT 

FLYING  TIME:  2 HRS  - 30/40  MIN  FILM  RUN 
FLIGHT  PATTERN:  TR  965974  TO  TS  980090 

RAPID  RAIL  MOVEMENT  OF  TRAIN  OF  APPROX.  25  CARS  HEADING  SW  VIC 
UR  115955  121723  SEP. 
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MSG  NO.  8 


MSG  NO.  10 

FROM:  S-2,  1BDE,  122207  SEP 
TO:  G-2 , 20  MECH  DIV 

CIVILIANS  REPORT  14  TRUCKS  WITH  16  CAA  BUMPER  MARKINGS  MOVING 
NORTHEAST  FROM  FALKENSTEIN  VICINITY  (UR  1395)  AT  121320  SEP. 

CARGO:  UNKNOWN. 

MSG  NO.  11 

FROM:  210  CA  CO,  122227  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIANS  HAVE  REPORTED  SEVERAL  LARGE  EXPLOSIONS  INSIDE  AGGRESSOR 
LINES  FROM  THE  DIRECTION  OF  TALTITZ  (TR  953913)  AT  APPROX  122020  SEP. 
CAUSE  UNKNOWN.  MECH  UNITS  OF  THE  IF  MECH  DIV  HAVE  BEEN  SIGHTED  IN 
THIS  AREA. 
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MSG  NO.  12 


FROM:  2240  MICO,  122349  SEP 
TO:  G-2 , 20  MECH  DIV 

AGGRESSOR  DOCUMENT  STRESSES  THE  REQUIREMENT  FOR  OPERATIONS  PLAN  TO 
EMPHASIZE  SPEED  AND  FIREPOWER  IN  ACHIEVING  THEIR  OBJECTIVE.  IF  MECH 
DIV  HAS  BEEN  DIRECTED  TO  SUBMIT  A "SPECIAL  REPORT"  BY  121300  SEP. 


MSG  NO.  13 

FROM:  G-2,  III  CORPS,  130017  SEP 
TO:  G-2,  20  MECH  DIV 

UNIDENTIFIED  OFFICER  FROM  GENERAL  STAFF  WAS  OVERHEAD  IN  EIBENSTOCK 
(UR  3096)  AT  121340  SEP  SAYING  THE  EXERCISE  HAS  PROVEN  SOME  MAJOR 
NEW  CONCEPTS  IN  THE  SUPPLY  PROCEDURES  OF  THE  16  CAA. 


MSG  NO.  14 


FROM:  S-2 , 2BDE,  130024  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIANS  INDICATE  THAT  SOME  TANK  UNITS  IN  THE  IF  DIV  MANEUVER  AREAS 
AS  OF  12  SEP  ARE  EQUIPPED  WITH  FORDING  EQUIPMENT.  THERE  ARE  ALSO 
AMPHIBIOUS  TANKS  INCLUDED  IN  THE  FORCE.  - 
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MSG  NO.  15 

FROM:  CDR,  20  GOP,  130105  SEP 
TO:  G-2 , 20  MECH  DIV 

US  UNITS  ASSIGNED  TO  THE  GOP  FORCE  (TR  970896)  HAS  HAD  MINOR  CONTACT 
WITH  AGGRESSOR  PATROLS  IN  THE  AREA  130037  SEP.  WHEN  CONTACT  WAS  MADE 
ENEMY  PA'iuOLS  WITHDREW . THERE  HAS  BEEN  NO  APPARENT  SIGNIFICANT 
CROSSING  INTO  FRIENDLY  TERRITORY. 


MSG  NO.  16 

FROM:  S-2 , 1 BDE,  130121  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIANS  REPORT  PATROLS  BEARING  IF  MECH  INSIGNIA  HAVE  BEEN  OPERATING 
ALONG  THE  EAST  SIDE  OF  THE  WEISSE-ELSTER  RIVER  ON  12  SEP.  THEY  HAVE 
QUESTIONED  RESIDENTS  EXTENSIVELY  ON  THE  OPERATIONS  OF  20  MECH. 

MSG  NO.  17 

FROM:  262  ASA  CO,  130123  SEP 
TO:  G-2,  20  MECH  DIV 

CG  OF  THE  19  CAA,  G/A  MIKHAYLOV,  DIMTRY , WITH  STAFF,  ENTERED  MANEUVER 
AREA  OF  34  MECH  DIV  AS  OF  121520  SEP.  PURPOSE  OF  VISIT  IS  TO  "REVIEW 
THE  PROGRESS  OF  THE  MANEUVERS".  ITINERARY  UNKNOWN. 

MSG  NO.  18 

FROM:  2240  MICO,  130133  SEP 
TO:  G-2,  20  MECH  DIV 

UNIDENTIFIED  TANK  UNIT(S)  OF  APPROX  BN  SIZE  LOCATED  IN  THE  VIC  OF 
AUERVACH  (UR  1699)  121321  SEP.  UNIT(S)  DISPOSITION  CENTERED  ON  THE 
MAJOR  ROAD  NETWORKS. 
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MSG  NO.  19 


FROM:  CDR,  20  GOP,  130135  SEP 
TO:  G-2,  20  MECH  DIV 

CONSTRUCTION  SOUNDS  HAVE  BEEN  HEARD  VIC  (TR  962900)  AT  130025  SEP. 
ELEMENTS  OF  THE  IF  MECH  DIV  HAVE  PREVIOUSLY  BEEN  REPORTED  IN  THIS 
LOCATION. 

MSG  NO.  20 

FROM:  CDR,  20  GOP,  130157  SEP 
TO:  G-2,  20  MECH  DIV 

FRIENDLY  CIVILIANS  HAVE  OVERHEARD  THAT  AN  AGGRESSOR  UNIT  OF  APPROX 
COMPANY  SIZE,  PART  OF  THE  211  REGT,  WILL  BE  MOVING  INTO  PREPARED 
POSITIONS  VIC  (TR  915912)  AT  APPROX  122015  SEP. 

MSG  NO.  21 

FROM:  G-2,  3 CORPS,  130207  SEP 
TO:  G-2,  20  MECH  DIV 

DESERTER  REPORTS  3 MECH  BN,  211  REGT,  LQCATED  VICINITY  (UR  0680) 

AS  OF  121710  SEP.  NO  FURTHER  INFORMATION  OF  TACTICAL  IMPORTANCE. 

i 

« 

MSG  NO.  22 

FROM:  2240  MICO,  130213  SEP 
TO:  G-2  20  MECH  DIV 

FRIENDLY  CIVILIANS  VIC  CROSS  ROADS  (UR  018985)  INDICATE  PASSAGE  OF 
TWO  GROUPS  OF  WHEELED  VEHCILES  AND  ONE  LARGE  GROUP  OF  TKS  OR  SP  GUNS, 
HEADING  SOUTH  AT  121116  SEP. 
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MSG  NO.  23 


FROM:  G-2 , 3 CORPS,  130217  SEP 
TO:  G-2,  20  MCH  DIV 

NEW  SEMI-AUTOMATIC  WEAPONS  AND  AT  WEAPONS  HAVE  RECENTLY  BEEN  ISSUED 
TO  THE  IF,  DIV,  CAUSING  CONFUSION  ON  THE  PART  OF  THE  AGGRESSOR  TROOPS 
DUE  TO  LACK  OF  TRAINING  IN  USE  AND  MAINTENANCE. 

MSG  NO.  24 

FROM:  CDR,  20  GOP,  130222  SEP 
TO:  G-2,  20  MECH  DIV 

AGGRESSOR  PSYCHOLOGICAL  WARFARE  UNITS  BROADCASTING  ALONG  THE  3 CORPS 
FRONT  NIGHTLY  EMPHASIZING  THE  COMBAT  POWER  OF  THE  AGGRESSOR  AND  THE 
FUTILITY  OF  THE  CAPITALIST  SYSTEM. 

MSG  NO.  25 

FROM:  CDR,  20  GOP,  130223  SEP 
TO:  G-2,  20  MECH  DIV 

GROUND  SURVEILLANCE  EQUIPMENT  INDICATE  LARGE  MOVEMENT  OF  TROOPS 
POSSIBLY  BN  STRENGTH  VIC  (TR  945913)  AT  130055  HTS . MECH  ELEMENTS 
OF  THE  IF  DIV  HAVE  BEEN  REPORTED  IN  THIS  VICINITY.  (B-3) 

MSG  NO.  26 

FROM:  G-2,  3 CORPS,  130229  SEP 
TO:  G-2,  20  MECH  DIV 

DOCUMENT  OF  16  CAA  INDICATES  THAT  THE  IF  DIV  ARTY  LOCATED  VIC  (UR  1081) 
120300  SEP  HAS  HAD  A HIGH  PERCENTAGE  OF  MISFIRES  AND  SHORT  ROUNDS. 
REPORTEDLY  COMMAND  ATTENTION  IS  BEING  DIRECTED  TO  THE  MATTER. 
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MSG  NO.  27 


FROM:  GOP,  20  MECH  DIV,  130234  SEP 
TO:  G-2,  20  MECH  DIV 

BODY  OF  AN  AGGRESSOR  SERGEANT  FOUND  IN  THE  WEISSE-ELSTER  RIVER  AT 
(UR  010843)  ON  130110  SEP.  INSIGNIA  INDICATED  HE  BELONGED  TO  THE 
210  ENG  REGT:  NO  OTHER  MARKINGS  OR  IDENTIFICATION. 

MSG  NO.  28 

FROM:  G-2,  3 CORPS,  130237 
TO:  G-2,  20  MECH  DIV 

SUBJECT:  G/D  TURGENEV,  I.,  CG  35  MECH  DIV 

CG  TURGENEV  HAS  BEEN  REPORTED  TO  BE  DEPRESSED  AS  THE  RESULT  OF 
PERSONAL  DIFFICULTIES  THAT  HE  IS  HAVING  AT  HOME. 


MSG  NO.  29 

FROM:  CDR,  20  SIG  BN,  130244  SEP 
TO:  G-2,  20  MECH  DIV 

SIGNAL  INTERCEPTED  AT  130010  SEP  ALERTS  THE  IF  MECH  DIV  TO  PREPARE 
FOR  MOVEMENT  BACK  TO  THE  HOMELAND  TO  SUPPORT  GARRISON  TROOPS  IN 
POSSIBLE  CIVILIAN  UPRISING. 


MSG  NO.  30 

FROM:  20  GOP,  130253  SEP 
TO:  G-2,  20  MECH  DIV 

OBSERVATION  OF  ENEMY  ACTIVITY  ALONG  EAST  SIDE  OF  WEISSE-ELSTER  RIVER 
INDICATES  PREPARATION  FOR  LAYING  PONTOONS. 
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MSG  NO.  31 


FROM:  S-2 , 2 BDE,  130301  SEP 
TO:  G-2,  20  MECH  DIV 

FRIENDLY  CIVILIAN  REPORTS  UNKNOWN  MECH  UNIT  ATTACHED  TO  IF  DIV  HAS  BEEN 
RELOCATING  SOME  CIVILIAN  RESIDENTS  OF  WARSCHITZ  (UR  0384)  AND  LEUBETHA 
(UR  0481)  TO  THE  AREA  EAST  OF  KLINGENTHAL  (UR  2082)  FOR  "THEIR  OWN 
PROTECTION." 

MSG  NO.  32 

FROM:  G-2,  3 CORPS,  130304  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIAN  REPORTS  FROM  THE  34  DIV  AREA,  INDICATE  AGGRESSOR  BUYING 
FOOD  IN  LARGE  QUANTITIES  FOR  WHICH  PAYMENT  IS  MADE  IN  MILITARY 
SCRIPT. 


MSG  NO.  33 

FROM:  G-2,  3 CORPS,  130314  SEP 
TO:  G-2,  20  MECH  DIV 

CORPS  LOCATED  MAJOR  AGGRESSOR  UNIT  CONCENTRATIONS  IN  THE  FOLLOWING 
AREAS: 

SCHONECK  (UR  1086) 

TIRPERSDORF  (UR  0591) 

TREUEN  (US  0902) 

UNIT  DESIGNATIONS  AND  STRENGTH  NOT  DETERMINED. 

MSG  NO.  34 

FROM:  S-2,  2 BDE,  130335  SEP 
TO:  G-2,  20  MECH  DIV 

CIVILIANS  REPORT  THAT  MOVEMENT  WITHIN  THE  IF  MECH  AREA  HAS  GENERALLY 
BEEN  ON  THE  INCREASE  SINCE  THEY  HAVE  OCCUPIED  THESE  POSITIONS. 


38 


MSG  NO.  35 


FROM:  IPW  SEC,  20  MECH  DIV,  130353  SEP 
TO:  G-2 , 20  MECH  DIV 

DESERTER  210  ENGINEER  REGT  STATED  THAT  210  HAS  BEEN  DRILLED  IN  THE 
RAPID  DEPLOYMENT  OF  PONTOON  SECTIONS  CAPABLE  OF  SUPPORTING  UP  TO  60 
TONS. 


MSG  NO.  36 

FROM:  CDR  20  GOP,  130414  SEP 
TO:  G-2,  20  MECH  DIV 

AGGRESSOR  PATROL  OF  APPROX  20  MEN  SIGHTED  VIC  (UR  009852)  AT  130226 
SEP.  NO  CONTACT  MADE. 


MSG  NO.  37 

FROM:  G-2,  3 CORPS,  130424  SEP 
TO:  G-2,  20  MECH  DIV 


MARTIAL  LAW  DECLARED  122130  SEP  WITHIN  THE  TOWNS  OF  OELSNITZ  (TR  995893), 
PLAUEN  (TR  970980),  FALKENSTEIN  (UR  135950),  AND  AUERBACH  (UR  158985) 


BY  AUTHORITY  OF  CG  16  CAA  FOR  THE  "DURATION  OF  THE  EXERCISE". 
MSG  NO.  38 


FROM:  CDR,  20  MECH  GOP  FORCE,  130437  SEP 
TO:  G-2,  20  MECH  DIV 


DESERTER  FROM  211  MECH  REGT,  IF  MECH  DIV  ENTERED  FRIENDLY  LINES 

I 

130115  SEP  IN  THE  21  ARMD  CAV  SQDN  SECTOR  OF  THE  GOP.  DESERTER 
STATED  HIS  REGT  HAD  MOVED  TO  FWD  ASSEMBLY  AREA  WITH  OTHER  UNITS  OF 
THE  IF  MECH  DIV  AND  HAS  MISSION  OF  ATTACKING  20  MECH  DIV.  DESERTER 
HAD  NO  IDEA  OF  LOCATION  OF  HIS  UNIT  AS  HE  LEFT  AT  NIGHT  AND  MADE 
HIS  WAY  TO  OUR  LINES. 
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MSG  MO.  39 


FROM:  CDR,  20  MECH  GOP,  130444  SEP 
TO:  G-2,  20  MECH  DIV 

UNITS  LOCATED  ON  THE  GOP  REPORT  A SHORT  (3  MIN)  EXCHANGE  OF  SMALL 
ARMS  FIRE  AT  (TR  915918)  130413  SEP.  CONTACT  FOLLOWED  BY  AGGRESSOR 
DISSENGAGEMENT. 

MSG  NO.  40 

FROM:  IPWSEC,  20  MECH  DIV,  130448  SEP 
TO:  G-2,  20  MECH  DIV 

INTERROGATION  OF  CAPTURED  AGGRESSOR  SABOTAGE  TEAM  NEAR  UNTERKOTZAU 
(QA  0680)  REVEALED  THEIR  MISSION  TO  BE  TO  INFILTRATE  US  LINES  AND 
CREATE  CONFUSION  BY  DESTROYING  POL  SUPPLIES  IN  THE  REAR  LINES. 


DISTRIBUTION 


ARI  Distribution  List 


4 OASD  (M&RA) 

2 HQDA  I DAM  I -CSZ) 

1 HQDA  (DAPE-PBR 
1 HQDA  (DAMA-ARI 
1 HQOA  (DAPE-HRE-PO) 

1 HQDA  (SGRD-IOI 
1 HQDA  (DAMI-DOT-C) 

1 HQDA  (DAPC-PMZ-A) 

1 HQDA  (DACH-PPZ  A) 

1 HQDA  (DAPE-HRE) 

1 HQDA  (DAPE  MPO-CI 
1 HQDA  (DAPE-DW) 

1 HQDA  (DAPE-HRL) 

1 HQDA  (DAPE  CPS) 

1 HQDA  (DAFD-MFA) 

1 HQDA  (DARO-ARS-PI 
1 HQDA  IDAPC-PAS-A) 

1 HQDA  (DUSA-ORI 
1 HQDA  (DAMO-RQR) 

1 HQDA(DASG) 

1 HQDA  (DAI  O-PI) 

1 Chief.  Consult  Div  (DA-OTSG).  Adelphi.  MD 
1 Mil  Asst.  Hum  Res.  ODDR&E,  OAD  (E&LS) 

1 HQ  USARAL,  APO  Seettle,  ATTN:  ARAGP-R 

1 HQ  First  Army,  ATTN:  AFKA-OI-TI 

2 HQ  Fifth  Army,  Ft  Sem  Houston 

1 Dir.  Army  Stf  Studies  Ofc.  ATTN:  OAVCSA  (DSP) 

1 Ofc  Chief  of  Stf,  Studies  Ofc 
t DCSPER,  ATTN:  CPS/OCP 
t The  Army  Lib,  Pentagon,  ATTN:  RSB  Chief 
1 The  Army  Lib,  Pentagon,  ATTN:  ANRAL 
1 Ofc,  Asst  Sect  of  the  Army  (R&D) 

1 Tech  Support  Ofc,  OJCS 
1 USASA,  Arlington,  ATTN:  IARD-T 

1 USA  Rsch  Ofc,  Durham,  ATTN:  Life  Sciences  Dir 

2 USARIEM,  Natick,  ATTN:  SGRD  UE  CA 

1 USATTC,  Ft  Clayton,  ATTN:  STETC  MO  A 
1 USAIMA,  Ft  Brags,  ATTN:  ATSU-CTD  OM 
1 USAIMA.  Ft  Bragg.  ATTN:  Marquat  Lib 
1 US  WAC  Ctr  & Sch,  Ft  McClellan,  ATTN:  Lib 
1 US  WAC  Ctr  & Sch,  Ft  McClellan,  ATTN:  Tng  Dir 
1 USA  Quartermaster  Sch,  Ft  Lee,  ATTN:  ATSM-TE 
1 Intelligence  Material  Dev  Ofc,  EWL,  Ft  Holabird 
1 USA  SE  Signal  Sch,  Ft  Gordon,  ATTN:  ATSOEA 
1 USA  Chaplain  Ctr  A Sch,  Ft  Hamilton,  ATTN:  ATSC-TE-RD 
1 USATSCH,  Ft  Eustis.  ATTN:  Educ  Advisor 

1 USA  War  College,  Carlisle  Barracks,  ATTN:  Lib 

2 WRAIR,  Neuropsychiatry  Div 
1 DLI,  SDA,  Monterey 

1 USA  Concept  Ana!  Agcy,  Bethesda,  ATTN:  MOCA-WGC 
1 USA  Concept  Anal  Agcy,  Bethesda,  ATTN:  MOCA-MR 
1 USA  Concept  Anal  Agcy,  Bethesda,  ATTN:  MOCA-JF 
1 USA  Artie  Test  Ctr.  APO  Seattle.  ATTN:  STEAC-MO-ASL 
1 USA  Artie  Test  Ctr,  APO  Seattle,  ATTN:  AMSTE-PL  TS 
1 USA  Armament  Cmd,  Redstone  Arsenal.  ATTN:  ATSK-TEM 
1 USA  Armament  Cmd,  Rock  Island,  ATTN:  AMSAR-TDC 
1 FAA-NAFEC,  Atlantic  City,  ATTN:  Library 
1 FAA-NAFEC,  Atlantic  City,  ATTN:  Hum  Engr  Br 

1 FAA  Aeronautical  Ctr,  Oklahoma  City,  ATTN:  AAC-44D 

2 USA  Fid  Arty  Sch,  Ft  Sill,  ATTN:  Library 
1 USA  Armor  Sch,  Ft  Knox,  ATTN:  Library 

1 USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-DIE 
1 USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-DT-TP 
1 USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-CD-AD 


2 HQUSACDEC,  Ft  Ord,  ATTN:  Library 

1 HQUSACDEC,  Ft  Ord,  ATTN:  ATEC-EX-E-Hum  Factors 

2 USAEEC,  Ft  Benjamin  Harrison,  ATTN:  Library 

t USAPACDC,  Ft  Benjamin  Harrison,  ATTN:  ATCP-HR 
1 USA  Comm-Elect  Sch,  Ft  Monmouth,  ATTN:  ATSN-EA 
1 USAEC.  Ft  Monmouth,  ATTN:  AMSEL-CT-HDP 
1 USAEC,  Ft  Monmouth,  ATTN:  AMSEL-PA-P 
1 USAEC.  Ft  Monmouth,  ATTN:  AMSEL-SI-CB 
1 USAEC,  Ft  Monmouth,  ATTN:  C,  Facl  Dev  Br 
1 USA  Materials  Sys  Anal  Agcy,  Aberdeen,  ATTN:  AMXSY— P 
1 Edge  wood  Arsenal,  Aberdeen,  A I IN:  SAREA— BL— H 

1 USA  Ord  Ctr  & Sch,  Aberdeen,  ATTN:  ATSL-TEM-C 

2 USA  Hum  Engr  Leb,  Aberdeen,  ATTN:  Library/Dir 

1 USA  Combat  Arms  Tng  Bd,  Ft  Banning,  ATTN:  Ad  Supervisor 
1 USA  Infantry  Hum  Rsch  Unit,  Ft  Banning,  ATTN:  Chief 
1 USA  Infantry  Bd,  Ft  Banning,  ATTN:  STEBC-TE-T 
1 USASMA.  Ft  Blits.  ATTN:  ATSS-LRC 
1 USA  Air  Def  Sch.  Ft  Bliss.  ATTN:  ATSA-CTD— ME 
1 USA  Air  Def  Sch.  Ft  Bliss,  ATTN:  Tech  Lib 
1 USA  Air  Def  Bd,  Ft  Bliss.  ATTN:  FILES 
1 USA  Air  Def  Bd,  Ft  Bliss,  ATTN:  STEBD-PO 
1 USA  Cmd  & General  Stf  College,  Ft  Leavenworth,  ATTN:  Lib 
1 USA  Cmd  & General  Stf  College,  Ft  Leavenworth,  ATTN:  ATSW-SE-L 
1 USA  Cmd  & General  Stf  College,  Ft  Leavenworth,  ATTN:  Ed  Advisor 
1 USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  DepCdr 
1 USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  CCS 
1 USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCASA 
1 USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCACO-E 
1 USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCACC  - Cl 
1 USAECOM,  Night  Vision  Lab,  Ft  Betvoir,  ATTN:  AMSEL-NV-SD 

3 USA  Computer  Sys  Cmd,  Ft  Betvoir,  ATTN:  Tech  Library 
1 USAMERDC.  FtBelvoir.  ATTN:  STSFB-DQ 

1 USA  Eng  Sch,  Ft  Belvoir,  ATTN:  Library 
1 USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  ETL-TD— S 
1 USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  STINFO  Center 
1 USA  Topographic  Lab,  Ft  Belvoir.  ATTN:  ETL-GSL 
1 USA  Intelligence  Ctr  & Sch,  Fi  Huachuca,  ATTN:  CTD-MS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATS— CTD-MS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TE 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI-TE X-GS 
1 USA  Intelligence  Ctr  A Sch,  Ft  Huachuca,  ATTN:  ATSI— CTS-OR 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI— CTD— DT 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI— CTD— CS 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  DAS/SRD 
1 USA  Intelligence  Ctr  & Sch,  Ft  Huachuca,  ATTN:  ATSI— TEM 
1 USA  Intelligence  Ctr  A Sch,  Ft  Huachuca,  ATTN:  Library 

1 CDR,  HQ  Ft  Huachuca.  ATTN:  Tech  Ref  Div 

2 CDR,  USA  Electronic  Prvg  Grd,  ATTN:  STEEP-MT-S 
1 CDR,  Project  MASSTER,  ATTN:  Tech  Info  Center 

1 Hq  MASSTER,  USATRADOC.  LNO 
1 Reeearch  Institute.  HQ  MASSTER,  Ft  Hood 
1 USA  Recruiting  Cmd,  Ft  Sherdian,  ATTN:  USARCPM-P 
1 Senior  Army  Adv„  USAFAGOD/TAC,  Elgin  AF  Aux  FW  No.  9 
1 HQ  USARPAC,  DCSPER,  APO  SF  96658,  ATTN:  GPPE-SE 
1 Stimeon  Lib,  Academy  of  Health  Sciences,  Ft  Sam  Houston 
1 Marine  Corps  Inst.,  ATTN:  Dean -MCI 
1 HQUSMC,  Commandant.  ATTN:  Code  MTMT  51 

1 HQUSMC,  Commandant.  ATTN:  Code  MPI-20 

2 USCG  Academy,  New  London,  ATTN:  Admission 
2 USCG  Academy,  New  London,  ATTN : Library 

1 USCG  Training  Ctr,  NY,  ATTN:  CO" 

1 USCG  Training  Ctr,  NY,  ATTN:  Educ  Svc  Ofc 
1 USCG,  Psychol  Res  Br,  DC.  ATTN:  GP  1/62 
1 HQ  Mid-Range  Br.  MC  Dot.  Ouantico.  ATTN:  PBS  Div 


1  US  Marine  Corps  Liaision  Ofc,  AMC,  Alexandria,  ATTN:  AMCGS— F 
1 USATRADOC.  Ft  Monroe.  ATTN:  ATRO-ED 
6 USATRADOC.  Ft  Monroe.  ATTN:  ATPR-AD 
1 USATRADOC.  Ft  Monroe.  ATTN:  ATTS-EA 

1 USA  Forces  Cmd,  Ft  McPherson,  ATTN:  Library 

2 USA  Aviation  Test  Bd,  Ft  Rucker,  ATTN:  STEBG— PO 
1 USA  Agcy  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Library 
1 USA  Agcy  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Educ  Advisor 
1 USA  Aviation  Sch,  Ft  Rucker,  ATTN:  PO  Drawer  O 

1 HQVJSA  Aviation  Sys  Cmd.  St  Louis.  ATTN:  AMSAV-ZDR 

2 USA  Aviation  Sys  Tilt  Art..  Edwards  AFB,  ATTN:  SAVTE-T 
1 USA  Air  Def  Sch,  Ft  Bliss.  ATTN:  ATSA  TEM 
1 USA  Air  Mobility  Rsch  & Dev  Lab,  Moffett  Fid.  ATTN:  SAVDL-AS 
1 USA  Aviation  Sch.  Res  Tng  Mgt,  Ft  Rucker,  ATTN:  ATST-T— RTM 
1 USA  Aviation  Sch,  CO.  Ft  Rucker.  ATTN:  ATST-D-A 
1 HQ.  OARCOM.  Alexandria.  ATTN:  AMXCD-TL 
1 HQ,  DARCOM.  Alexandria.  ATTN:  CDR 
1 US  Military  Academy,  West  Point,  ATTN:  Serials  Unit 
1 US  Military  Academy,  West  Point.  ATTN:  Ofc  of  Milt  Ldrshp 

1 US  Military  Academy,  West  Point.  ATTN:  MAOR 
1 USA  Standardization  Gp.  UK.  FPO  NY,  ATTN:  MASE-GC 
1 Ofc  of  Naval  Rsch,  Arlington.  ATTN:  Code  452 

3 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  458 
1 Ofc  of  Naval  Rsch.  Arlington,  ATTN:  Code  450 
1 Ofc  of  Naval  Rsch.  Arlington.  ATTN:  Code  441 

1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Acous  Sch  Div 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L51 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L5 
1 Chief  of  NavPers.  ATTN:  Pers-OR 
1 NAVAIRSTA.  Norfolk.  ATTN:  Safety  Ctr 
1 Nav  Oceanographic.  DC.  ATTN:  Code  6251 , Charts  & Tech 
1 Center  of  Naval  Anal,  ATTN:  Doc  Ctr 
1 NavAirSysCom,  ATTN:  AIR-53 IX 
1 Nav  BuMed  ATTN:  713 
1 NavHelicopterSubSqua  2,  FPO  SF  96601 
1 AFHRL  (FT)  William  AFB 
1 AFHRL  (TT)  Lowry  AFB^ 

1 AFHRL  (AS)  WPAFB.  OH 

2 AFHRL  (DOJZ)  Brooks  AFB 

1 AFHRL  (DOJN)  Lackland  AFB 
1 HQUSAF  (INYSD) 

1 HQUSAF  (DPXXA) 

1 AFVTG  (RD)  Randolph  AFB 

3 AMRL  (HE)  WPAFB.OH 

2 AF  Inst  of  Tech.  WPAFB,  OH,  ATTN:  ENE/SL 
1 ATC  (XPTD)  Randolph  AFB 

.1  USAF  AeroMed  Lib.  Brooks  AFB  (SUL-4),  ATTN:  DOC  SEC 
1 AFOSR  (NL),  Arlington 

1 AF  Log  Cmd,  McClellan  AFB.  ATTN:  ALC/DPCRB 

1 Air  Force  Academy,  CO,  ATTN:  Dept  of  Bel  Sen 
5 NavPers  St  Dev  Ctr,  San  Diego 

2 Navy  Med  Neuropsychiatric  Rsch  Unit,  San  Diego 
1 Nav  Electronic  Lab.  San  Diego,  ATTN:  Res  Lab 

1 Nav  TrngCen,  San  Diego,  ATTN:  Code  9000-Lib 
1 NavPostGraSch,  Monterey,  ATTN:  Code  55Aa 
1 NavPostGraSch,  Monterey,  ATTN:  Code  2124 
1 NavTrngEquipCtr,  Orlando,  ATTN:  Tech  Lib 
1 US  Dept  of  Labor,  DC,  ATTN:  Manpower  Admin 
1 US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1 Nat  Bur  of  Standards,  DC,  ATTN:  Computer  Info  Section 
1 Nat  Clearing  Houso  for  MH— Info,  Rockville 
1 Denver  Federal  Ctr.  Lakewood,  ATTN:  BLM 
12  Defenae  Documentation  Center 

4 Dir  Psych,  Army  Hq,  Russell  Ofcs,  Canberra 

1 Scientific  Advsr,  Mil  Bd,  Army  Hq,  Russell  Ofcs,  Canberra 
1 Mil  and  Air  Attache,  Austrian  Embassy 

1 Centre  de  Recherche  Des  Facteurs,  Humaine  de  la  Defense 
Nationale,  Brussels 

2 Canadian  Joint  Staff  Washington 

1 C/Air  Staff,  Royal  Canadian  AF,  ATTN:  Pars  Std  Anal  Br 

3 Chief,  Canadian  Oef  Rsch  Staff.  ATTN:  C/CRDS(W) 

4 British  Def  Staff,  British  Embassy,  Washington 


1 Daf  & Civil  Inst  of  Enviro  Medicine,  Canada 
1 AIR  CRESS,  Kensington,  ATTN:  Info  Sys  Br 
1 Mllitaerpsykologisk  TJeneste,  Copehegen 
1 Military  Attache.  French  Embassy,  ATTN:  Doc  Sec 
1 Medecin  Chef.  C.E.R.P .A.— Arsenal,  Toulon/Naval  France 
1 Prin  Scientific  Off,  Appl  Hum  Engr  Rsch  Div.  Ministry 
of  Defense.  New  Delhi 

1 Pers  Rsch  Ofc  Library,  AKA,  Israel  Defense  Forces 
1 Minister  is  van  Defensie,  DOOP/KL  Afd  Sociaal 
Psychologische  Zaken,  The  Hague,  Netherlands 


